1. Field of the Invention
This invention relates to the use of EEG recordings to detect brain responses for task-relevant stimuli.
2. Description of the Related Art
Electroencephalography (EEG) is the neurophysiologic measurement of the electrical activity 10 of the brain recording from electrodes 12 placed on the scalp as depicted in FIG. 1. The EEG signals contain data and patterns of data associated with brain activity. Cognitive neuroscience techniques use a multi-channel spatial classifier 14 to analyze the EEG signals to infer the existence of certain brain states 16. These techniques are used to study the brain and to perform clinical diagnosis. These techniques may also be used to detect significant brain responses to task-relevant stimuli.
The detection of such a response may be fed back or used in some manner in conjunction with the task. For example, the brain response could be used in a classification system to detect and classify visual, auditory or information stimuli, a warning system to detect potential threats, a lie detector system etc. The detection of a significant brain response does not classify the stimulus but raises an alert that the user's brain has responded in a significant way. Some of the alerts will be false positives or ‘false alarms’ but if the system is configured properly to reduce the false alarm rate the detected brain responses may be very useful. Current systems try to improve the signal-to-noise ratio (SNR) of the EEG signals to reduce the false alarm rate at a given detection probability.
EEG signals represent the aggregate activity of millions of neurons on the cortex and have high time-resolution (capable of detecting changes in electrical activity in the brain on a millisecond-level). Evidence suggests significant amplitude differences between trial-averaged EEG responses triggered by task-relevant stimuli versus trial-averaged EEG responses triggered by neutral stimuli. The benefit of integrating EEG responses across multiple trials is to suppress the task-unrelated background EEG and project out the task-relevant EEG saliently, i.e. improve the signal-to-noise ratio.
One example of a trial-averaged system is described by Tan in US Pub No. 2007/0185697 entitled “Using Electroencephalograph Signals for Task Classification and Activity Recognition”. To train the classifier, Tan collects labeled EEG data while a person is performing a task and divides the data into overlapping time windows e.g. 2 second windows with a 50% overlap over a 12 second interval. The time dimension is then removed from each of the time windows. Features are extracted from each dimensionless window, aggregated and then pruned to form a feature set for a single classifier. The trained classifier is used to classify brain states in unlabeled EEG by dividing the data into overlapping time windows and then processing each window to remove the time dimension. The ‘pruned’ features are extracted for each window and presented to the same classifier. The results are then averaged over time to improve SNR.
In other contexts, the event or task stimulus cannot be trial-averaged. For example, the stimulus may occur only once (“single-event”). Alternately, the application may need to classify the brain response in real-time. In Tan, the user repeats the task for several seconds alloying the classifier to process multiple 2-3 second windows before classifying the task. As Tan's method is directed at evaluating the effectiveness of a computer-user interface, the stimulus can be repeated and a delayed classification is acceptable.
The biggest challenge of single-event real-time detection is to overcome the low SNR problem imposed by event-unrelated background EEG responses that usually have larger amplitude than event-related responses and could completely obscure the later. Recent advances in adaptive signal processing have demonstrated significant single trial detection capability by integrating EEG data spatially across multiple channels of high density EEG sensors (L. Parra et al, “Single trial Detection in EEG and MEG: Keeping it Linear”, Neurocomputing, vol. 52-54, June 2003, pp. 177-183, 2003 and L. Parra et al, “Recipes for the Linear Analysis of EEG”, NeuroImage, 28 (2005), pp. 242-353)). The linear (LDA) classifier provides a weighted sum of all electrodes over a predefined temporal window as a new composite signal that serves as a discriminating component between responses to target versus distractor stimuli.
A rapid serial visual presentation (RSVP) system for triaging imagery is an example of a single-event system in which cognitive neuroscience techniques for classifying brain states have been employed (A. D. Gerson et al “Cortical-coupled Computer Vision for Rapid Image Search”, IEEE Transaction on Neural Systems and Rehabilitation Engineering, June 2006). The US military and intelligence agencies generate enormous amounts of imagery that require visual examination to identify possible targets of interest for further inspection. In the RSVP system, electrodes are placed on an analyst's scalp, image clips are displayed to the analyst at a rate of approximately 10 per second and a multi-channel LDA classifier is employed to classify the brain response to the presentation of each image. If a significant response is indicated, the system flags the image clip for closer inspection.
For both the trial averaged approach and the spatially integrated single trial approach, testing revealed that the event-related EEG response triggered by target detection is most prominent at a certain critical time period after stimulus onset. Thorpe et al “Speed of processing in the human visual system”, Nature Vol. 381, pp. 530-532, 6 Jun. 1996 found that the trial averaged ERP generated on target versus distractor trials diverge sharply at 150-200 milliseconds after stimulus onset for a go/no-go image categorization. Para et al applied the LOA classifier in the RSVP on EEG data in a predefined temporal window centering around the time where the target trial averaged ERP most sharply diverged from the distractor trial averaged ERP. Similarly, Gerson used a training window between 400-500 ms following stimulus onset to extract training data. Gerson also recommended using multiple classifiers with different training window onsets to boost triage performance. The training window onsets ranged from 0 to 900 ms in steps of 50 ms with window durations of 50 ms. Once these classifiers were trained, the optimal weight of these classifier outputs was found using logistic regression to discriminate between target and non-target images.